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1. INTRODUCTION 

The high dimensional information processing through neural network is emerging as a fascinating 
but challenging field of research in the second generation neurocomputing. The recent researches in high 
dimensional neural networks have established their superiority [1], [2], [3], [4] over real-valued or first 
generation neural networks. Although, real-valued neural networks (RVNN) have been used to process high 
dimensional data, but the network needs to employ too many neurons resulting huge structure and slow 
learning. The RVNN can also not process phase information during learning and generalization of mapping 
on the plane [2], [5], [6]. The complex-valued neural networks (CVNN) can promptly process two 
dimensional information with phase as a single number, which leads to a drastic reduction in the complexity 
of the network along with better performance. But, neural network of three dimensional information still 
needs an exhaustive investigation. The applications with three dimensional information are popular in 
computer vision, robotics, biometrics, bioinformatics etc. The few researchers attempted machine learning 
with three dimensional information considering it as a vector [7], [8]. The corresponding learning algorithms 
have restrictions on weight matrix and a vector does not provide freedom like a complex number, as in 
CVNN [8]. Thus, it is very demanding to have neural network, which may promptly process different high 
dimensional parameters as numbers and can be simply incorporated in various applications of intelligent 
machine design, like CVNN [9], [2]-[3]. In the enhancement of higher order number systems the complex 
numbers (2D), quaternions (4D), octaves (8D), sedenions (16D) were developed by mathematicians in the 
past but there is no number system in three dimensions [10]. The researches [1-3, 6Jalso elaborate that the 
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CVNN has outperformed over RVNN even for real-valued problems, therefore we propose to exploit 
quaternions in neural network to process three dimensional problems. 

The neurocomputing with high dimensional number systems will definitely overcome from learning 
and generalization of huge conventional neural network and lead to lower complexity. The quaternion is one 
of the hypercomplex number introduced by Iris mathematician Hamilton [11] which has been extensively 
employed in the field of quantum mathematics, physics, computer graphics, signal processing and 
control [12-13, 18-17]. This number system has recently popped up in neural network through quaternionic 
neurons, as complex or real -valued neurons, to develop efficient machine learning in higher dimensions. 
Few attempts have been made in this direction, the orthogonal decision boundary of single quaternionic 
neuron has been utilized to solve 4-bit parity problem in [14]; quaternionic MLPs proposed in [15] has the 
problem of existence of singularities; quaternion-valued algorithms are proposed for adaptive filtering [18]. 
[17]; a basic work for quaternionic-valued neural network with sigmoidal activation function is presented in 
[18, 19]. In this paper, we present not only simple, straightforward, but potential machine learning algorithm 
for sufficient general structure of the quaternionic domain neural network (QDNN) but also demonstrate the 
evaluation over the wide spectrum of applications, like function approximation, motion interpretation and 
recognition in space. The parameters in QDNN, like synaptic weights, biases, inputs-outputs signals and 
internal potentials are quaternions and represented as quaternion matrix, in multilayer neural network. 
Although, Hamilton proposed quaternionic numbers (q = qo + qi + q2j +q3k ) for 4D number system 
[11], but it can also bring into play any 3D information in the space after equating its real part zero. The 
presented learning algorithm based on the error backpropagation for QDNN can efficiently solve any typical 
class of problems in 3D and 4D. The analytic [1, 8] or split type [1], [5], [7] activation functions have been 
chosen for complex-valued neuron which have their own issues concerning boundedness and analyticity. 
Therefore, selection of suitable activation function for neuron dealing with quaternion is one of the important 
concerns. The split type function may not be appropriate when analyticity is concerned, similarly the analytic 
function is not suitable when the singularity arises. The presented QDNN prefer boundednes over analyticity 
and use “split-type” activation function. The QDNN outperform with lesser number of neurons and faster 
learning where conventional real-valued neural network (RVNN) lacks. The quaternionic-valued neural 
network (QDNN) has an ability to learn and generalize 3D motion of objects and recognition of the point 
cloud object, but RVNN cannot, because QDNN has ability to capture and maintain phase information of 
each point during the learning and generalization. 

This paper investigates the general structure of QDNN with learning algorithm through simulation 
on various benchmark problems of different sphere of influence. The sections and sub-sections of the paper 
are organized as follows: The section 2, presents a complete machine learning framework with pseudo code 
of learning in quaternionic domain. Section 3 evaluates the learning and generalization capability through 
function approximations, linear transformations and 3D face recognition. Section 4 presents the final 
conclusion and future scope of the work. 


2. MACHINE LEARNING IN QUATERNIONIC DOMAIN 

A quaternionic number system is the straightforward extension of real and complex number system, 
where four components are incorporated in single number; the first component acts as real and other three as 
imaginary with unit vectors (i, J, k). These imaginary components overlie on the axes in three dimensional 
space [11, 12]. A quaternionic variable (q = qo + qi + q2j + G3k ) consists of a real component (qq) and 
three imaginary components (q1, q2, q3). Its bases (i,j, k) are orthogonal special vectors. Thus, they follow 
the properties as i? = j? = k? = —1 and cross product properties asi x j = —(j x i) = k, j x k = -(k x 
J) =t, kxi=-(ixk) =j. Ina prominent representation, a quaternion (q) can be expressed in the form 
of a matrix (quaternionic matrix): 


do 41 q2 43 


q = —~% qlo —43 q2 (1) 
—d2 93 qo 74| 
—43 —G2 qı qdo 


The bold type letter denotes quaternionic variable or quaternionic matrix. The conjugate of 
quaternionic variable (q* = qo — qi — q2j — qK) is similar to complex conjugate and the conjugate of 
quaternionic matrix denotes the transpose of the quaternionic matrix, defined as: 
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do 4 2 Q3]! do —q1 742 —]3 


q = q! _|~% Go 743 q2 _|/% do 43 42 (2) 
—G2 43 qo 4 q2 —q43 qo % | 
—43 —q2 qı qdo d3 42 qi qo 


The machine learning optimization technique incorporates the basic operations of quaternion 
algebra [11, 12]. The addition and subtraction of two quaternionic matrices q and r cab be obtained simply 
as matrix operations. The multiplication of two quaternionic matrices q and r does not follow the 
commutative property (qr # rq). The inner product of two quaternionic matrices q and r is expressed by: 


417% qoro 9373 q2%2 (3) 
G2T2 9373 qoro qra | 
9373 {22 411% qoro 


—42 43 qdo Yo P 


do 41 42 doro 91% q2%2 4313 
—41 qo —43 % o a x 

r = 
—q43 q2 qı n ri 


The norm of quaternionic matrix q is expressed as: 


1 (4) 
lal = 5 2 diag(qq’) = |q + qf +q +95 


2.1. Learning in Quaternionic Domain Neural Networks 

Let a three layer (L — M — N) QDNN possesses L inputs; M and N quaternionic neurons in hidden 
and output layers respectively. All inputs, outputs, weights and biases signals are considered as quaternionic 
matrices, as represented in Eq. (1). The derivation of optimization technique incorporates the basic operations 
of quaternion algebra which present the compact and the generalized derivation of the backpropagation 
algorithm (QDBP) of three-layer network. The bold letters denote the quternionic matrix or matrix containing 
quaternionic matrices as elements. 


2.1.1. Forward Pass 
Let us consider 17, Ij‘, 17, IZ be the 4D quaternionic input of It” (l = 1 ... L) neuron in the input layer 
of the network. The quaternionic input can be expressed as a quaternionic matrix (T): 


L È R K 


ra e a a (5) 
J- E g -r| 
-7 -F E if 
The matrix of inputs (I) at the input layer of the network is defined by: I = [L I, Iz + L]! (6) 


The initialization of synaptic connection weights Wm; and Spm are defined for lt” input to mt” (m = 
1...M) hidden neuron pair and for m” hidden to n(n = 1...N) output neuron pair of network 
respectively. These weights are presented in quaternionic matrices containing a real and other three 
imaginary components as follows: 


r x y Z 
Wm Wm W Wmi 


ml 
-Wn Wm Wm Wmi 
-Whi —Wint Wai Wmi 
Snm Sam Aon Sam 
Snm = n: Sum Sam Siam i 
—Shm Snm Sam —Snm 


Z ae x 
—Snm Snm Snm Snm 


Similarly, the initialization of biases a,, and B,, are defined for m*” hidden and nt” output neuron of 
network: 
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-An Gn a, -ah 
-až, —a}, a ath 
n Pa Br Bn 
g -|E Bi -pk Br vo 
" |-Bn Pa Ba Bx 


-ph -Ba BR Ba 


The internal potential matrix U, for neurons (1 .. M) at hidden layer of the network is defined as: 


U=Wi+a. (11) 


U, W11 W12 W13 + 

U, W21 W22 W23» 

U, |= Wat W32 Wsi + as (12) 
Uy WWW u3 Wt 


where, elements of weight matrix W contains corresponding weights between input to hidden neurons and 
elements of bias matrix æ contains biases of hidden neurons. Let f be an activation function and f’ be its 


derivative. The output matrix (O) is obtained by split-type activation function over internal potential matrix 
(U) at hidden layer: 


O=f(U). (13) 
[0; 0z ... Om: Oy)’ = [f (U1) f(U2) -~ fm) fU. (14) 


where, 


FU) FU) fur) FUR) 
B _ |f(-Un) fUn) f(—UZ) f(U%) (15) 
Om =FUm) = |u) FUR Fur) fuz) 
f(-U2) f(-U2) FU) f(t) 


The internal potential matrix V at output layer of the network is defined as: 


V = SO + B. (16) 
S11 S12 $13 -Sım pO; pı 
S21 S22 S23 »+:S2m || 02 f- 
sa oe Sees 3M || O3|+| Bs |. (17) 
Seperate Om By 


where, elements of weight matrix S possesses strength of synaptic connections between hidden and output 
neurons and column vector P possesses all quaternionic biases of respective output neurons. The output 


matrix (Y) is obtained by applying split-type activation function over internal potential matrix (V) at the 
output layer: 


Y =f(V). (18) 


[Y] Y> Yas Yy] = If) f2) f(V3) = f(Vy)I’. (19) 
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where, 


F F FOR). FVA 
SMD fh) FVA FVD) 
FOR FO F FET (20) 
FVZ FER FU FWP) 


Yn = fn) = 


2.1.2. Backward Pass 

In order to develop a QDNN based learning machine, we present the derivation of the error 
backpropagation learning algorithm in quaternion domain (QDBP) through minimization of average mean 
square error (E) of the network: 


4N 
1 
E= = diag(diagonalmatrix(e)diagonalmatrix(e*)) 


n=1 
€i ei 
i 4N e> e; 
= = diag | diagonalmatrix | | €3 | | diagonalmatrix | | e3 
n=1 : 
en eN 
ei 0 ei 0 
í 4N e- e5 
= = diag e3 7 e3 : ; (21) 
n=1 
0 e€v4lO en 


where, * denotes quaternionic conjugate (as defined in Eq. (2)) and the output error matrix (e) presents the 
difference between actual (Y) and desired (Y”) output at output layer, defined as: 


e=Y- P. (22) 
ei Y, y? Yı B y? 
€2 Y, Y- Y, = Yo 
e3 |=|Y3|-|y3|=]|Y;- Y; | (23) 
end LYnd Ly?! (Y-Y? 


The update equations of weight and bias matrices are obtained by employing a gradient decent 
optimization approach on MSE, mean square error (E). The weight update matrix (AS) between hidden- 
output layers and bias update matrix (AB) at the output layer of the network are presented as follows: 


AB, e,0 f'V;) 
AB» n e2 O f'(V2) 


AB = | AB3 | = NIe © f'U3) | (24) 
Apn ey O f' Vw) 
AS11 AS12 AS3--ASiy e, © f'(V,) 17/077" 
AS21 ÂS22 AS23 +-ASzy n | &2 © f'(Vz) || 03 

AS = AS31 AS32 AS33 «-AS3mu = N e> © f'(V3) 0; f (25) 
AS ASy2ASy3°"ASyy en O f' Vn )ILOn 
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where, 7 € R* denotes a learning rate and © denotes element-wise multiplication of two 
quaternionic matrices (as defined in Eq. (3)). Similarly, weight update matrix (AW) between input-hidden 
layers and bias update matrix (Aœ) at hidden layer of the network are presented as follows: 


Aa, As, AS;2 AS;3+-AS;y 4 e O f'a) f' (U) 
Aa, n AS>, ASz7 AS23 AS>m4 e, © f'(V2) f' (U) 

Aa = | Aq; | = N AS31 AS32 AS33 :-AS3mu e3 © f'W3) O f' (U3) i (26) 
Aady ASyiASy2ASy3°"ASyui Ley © f'n) f' Un) 


Aw,, AW12 Aw,3 = AW, 
Aw, AW27 AW>3 «AW a, 
AW =| AW31 AW32 Aw33 ---AW3, 


AW yi AWy24W m3 "AW m, (27) 
AS11 AS12 AS;3-- AS; M d e, O f'(V;) f' (U) D ` 
n AS 21 ASp7 AS23--ASzy e- © f' (V2) f'W2) 2 
= | | | 48314832 Asa3-Asam| | es © f(s) | [O| f'Us) | || 


Asy1ASy2ASy3°"ASyyt Ley © f'Vy) f'n) I, 


2.2. Learning algorithm in quaternionic domain 

For the sake of simplicity and better understanding, we further present an algorithm 
QDNN_TRAIN(.) for training of quaternionic domain neural network (QDNN), which is elaborated by 
procedures QDNN_INIT(.), QDNN_FORWARD(.) and QDNN_BACKWARD(.). The learning and 
generalization ability of a three-layered neural structure is obtained through optimization of mean square 
error. The procedure QDNN_INIT(.) randomly initializes the weight and bias matrices in considered 
network. It calls the RANDOM_QM(a, b) procedure which randomly generates the quaternionic matrix of 
each interconnection weight and bias of neuron in the range from a to b. The QDNN_FORWARD(.) 
procedure is intended to implement forward pass of QDNN, hence generate internal potentials (U,V) and 
hence outputs (O,Y) matrices at respective layers. The ACTIVATION _FUNCTION(.) limits the output of 
corresponding neuron of the network. For updates weight and bias matrices, QODNN_BACKWARD(.) is 
developed for the backward pass of QDNN. All required procedures are presented in pseudo code are as 
follows: 


procedure QDNN_TRAIN(/, Y?,7, €) 
begin 
QDNN_INIT(L, M, N); 
while Er > edo 
for i <— 1untilS = length (1) do 
U,0O,V,Y — QDNN_FORWARD(W,a, 5S, B,D; 
e- Y-— YP; 
E; < — AN, diag (diagonalmatrix(e)diagonalmatrix(e*)); 
QDNN_BACKWARD(W, a, U, O,S, B,V,Y,n, e) 
Er — = Dies Ej; 
end 
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procedure QDNN_INIT(L, M,N) 
begin 
form < 1until M do 
for! <— 1 until L do 
Wmi | RANDOM_QM(a, b); 
a, <— RANDOM_QM(a, b); 
forn < 1 until N do 
form < 1 until M do 
Snm <— RANDOM_QM(a, b); 
Bn <— RANDOM_QM(a, Db); 
end 


procedure QDNN_FORWARD(W, a; 5S, B, I) 
begin 

U «+ Wi+a; 

O < ACTIVATION_FUNCTION(U); 

V -S0+ B£; 

Y — ACTIVATION_FUNCTION(V); 
end 


procedure QDNN_BACKWARD(W, «æ, U, O,S, B,V,Y,7,e) 
begin 

AB + (n/N)e © DER_ACTIVATION(V); 

AS < (n/N)(e © DER_ACTIVATION(V))O*’ ; 


Aa — (n/N)(S™(e © DER_ACTIVATION(V))) © DER_ACTIVATION(U); 
AW <— (n/N)((S™(e © DER_ACTIVATION(V))) © DER_ACTIVATION(U))I*’; 


BB+ AB; 

S - S + AS; 

q — a + Ag; 

W -W + AW; 
end 


procedure RANDOM_QM(a,b) 


begin 
qo — la+ (b—a)]RAND(1); 
qı — [a+ (b—a)|RAND(1); 
q2 < |a+(b—a)|RAND(1); 
q3 — [a+ (b — a)|RAND(1); 
do 41 q2 43 
qe —41 qlo — 43 q2 . 
—q2 q3 qo qi 
—q3 —q2 qı qdo 
end 


procedure ACTIVATION_FUNCTION(q) 
begin 
Q = fq); 


end 


3. PERFORMANCE EVALUATION OF LEARNING MACHINE THROUGH BENCHMARK 
PROBLEMS 
In this section, we evaluate the effectiveness of learning machine through a wide spectrum of 
benchmark problems: function approximations, linear transformations, and 3D face recognition. The 
components of all quaternionic weights and biases are randomly initialized in the range -1 to 1. The 
quaternionic variable qg = 1 +i +j + kis assumed as bias input and the hyperbolic tangent function is used 
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as activation function. A comparative performance between first generation ‘real-valued neural network’ and 
second generation ‘quaternionic-valued neural network’ with respective algorithms real-valued 
backpropagation (RVBP) and quaternionic-domain backpropagation (QDBP) is thoroughly evaluated for 
function approximations by statistical parameters like error variance, correlation, and AIC [20]. Another class 
of benchmark problems, the learning of linear transformations (rotation, scaling, and translation and their 
combinations), is promising one as training is performed through a few sets of point lying on the line and 
trained network is able to generalize over complicated 3D geometric structures. In last subsection, two 
primary experiments are presented for 3D face recognition; surely it will be stepping stone for prospective 
researchers to extend this novel technique over a large data set. In last two experiments, each point is 
represented by a quaternion which contains intended components along with phase information embedded 
within a number, therefore RVNN is not able to perform such experiments. 


3.1. Function Approximations 
3.1.1. The Lorenz System 

The dynamics of the Lorenz system [21] is presented by the system of three differential equations 
which shows the chaotic behavior depending on its parameter values. 


dx/dt=o(y-x) 
dy/dt=x(p-z)-y 
dz/dt=xy-Bz (28) 


where, the symbols g, p and f are parameters of the Lorenz’s system. On the basis of its parameters 
(o = 15, p = 28 and f = 8/3), this system (Eq. (28)) generates 6537 terms of the time series with initial 
condition (x = 0.7, y = 0.1, z = 0.1) using fourth order Runge-Kutta method. Each term can be considered 
in the form of quaternionic input as 0 + xi + yj + zk. Further, the normalization is performed in the range 
from -0.8 to 0.8. The first 500 terms of the time series have been used for training and rest for testing of 
three-layered RVNN (3-11-3) and QDNN networks (1-3-1) separately. Experiments demonstrate that the 
second network requires a lesser number of training cycles to achieve the desired MSE, as presented in Table 
1. Figure. 1 shows the testing results of the networks trained by QDBP for prediction of time series of Lorenz 
system. Table 1 demonstrates the significant outperformance of QDNN in terms of network topology, 
training cycles, testing MSE, error variance, correlation and AIC. 





Figure. 1. 3D plot of the Lorenz system tested by the QDNN network trained through QDBP 


Table 1. Comparison of training and testing performance for Lorenz system 


Neuron Type Real-valued Quaternionic-valued 
Algorithm RVBP QDBP 
Network Topology 3-11-3 1-3-1 

MSE Training 0.0015 0.0006 
Average Epoch 15000 9000 

MSE Testing 0.0042 0.0012 

Error Variance 0.0026 0.0009 
Correlation 0.87327 0.9323 

AIC -6.3329 -7.4503 
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3.1.2. The Chua’s Circuit 

Chua’s circuit is the simplest autonomous electronic circuit containing registers, capacitors and 
inductors that exhibit the chaotic behavior under specific parametric conditions [22]. This circuit satisfies the 
chaotic criterion which contains one or more non-linear elements, one or more active registers and three or 
more energy storage devices. It uses the one chua’s diode as non-linear element, one locally active register 
and two capacitors and one inductor as energy storage devices. The dynamics of Chua’s circuit are governed 
by three state equations as 


ae aly- x- he) 

dy _ 

a ee (29) 
dz 

a ye 


where, h(x) presents the electrical response of non-linear register defined as 
h(x) = mix +5 (mo — m,)(lx + 1| — Ix — 11) 


and a, P, Y, Mo and m; are the constant parameters. The symbols x, y and Z are voltages across two 
capacitors and an inductor respectively, and their combinations show the chaotic attractor in three 
dimensions. The double scrolled chaotic attractor [22] is obtained with the parameters a = 15.6, 6B = 28, 
y = 0, Mmo = —1.143 and m, = —0.714. The chaotic time series has been obtained from the simulation of 
the system (Eq. 29) with time step 0.1 Sec and initial voltages x=0.1, y = 0.1 and z = 0.1 by using fourth 
order Runge-Kutta method. The normalization of input-output imaginary quaternions is done in -0.8 to 0.8 
(real part is zero and imaginary parts (x, y, Z) present corresponding voltages). A time series containing 500 
terms obtained from simulated system has been used to train RVNN and QDNN. The training results of both 
networks, in Table 2, demonstrate that QDNN trained by the QDBP algorithm requires a significantly smaller 
number of average epochs to achieve the threshold training error than RVBP. The next 500 terms of that time 
series have been tested through networks trained by both algorithms. Figure. 2 shows the 3D patterns of 
desired and actual data for chaotic behavior of Chua’s circuit. The testing results shown in Table 2 in terms 
of error, variance, correlation, and AIC again infer the superiority of QDNN over real-valued neural network. 








Figure 2. Testing result of QVNN network trained by QDBP for Chua’s circuit 


Table 2. Comparison of training and testing performance for Chua’s circuit 


Neuron Type Real-valued Quaternionic-valued 
Algorithm RVBP QDBP 
ates LE 1-3-1 

MSE Training 0.0012 0.0008 

Average Epoch 10000 7000 

MSE Testing 0.0025 0.0017 

Error Variance 0.0020 0.0008 
Correlation 0.9734 0.9874 

AIC -6.5332 -7.0101 
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3.2. Linear Transformations 

In order to evaluate the performance of QDNN, we have considered a three layer neural structure 
(2-M-2). This section presents the learning of linear transformations (rotation, scaling, and translation and 
their combinations) by QDNN through a few sets of points on the line and generalization over complicated 
3D objects. Each quaternionic variable q; = 0 + x;i + yij + zik undergoes a transformation function (T) and 
correspondingly yields a transformed quaternionic variable q; = 0 + x;i + y;j+z;k represented in the 
quaternionic matrix as follows: 


qi =T(qi) =aqitb (i= 1,2,3, ...np) 


0 Xi Yi Zi Vi 0 by b, 
—=xi 0 -=z $2 zs “4, Sas | —Z; epee by —b, by 
“Vi Zi xi ay Tax||-yi Zi =x; by bz 0 -b 
—Zi —y; 0 7 ay ox ; Vi x b, —by by 0 


where n, denotes the number of points that lies on the surface of 3D objects and a and b are quaternions 


such that norm of ai.e. |la|| =./0? + a? + aż + aĉ denotes the scaling factor. Argument of a yields 
rotation in q while b performs translation of 3D object in the distance (||b||). The combinations of 
transformations facilitate the viewing of 3D objects from different orientations, interpretation of their motion, 
etc. 

For training on a three layered 2-6-2 QDNN, all experiments consider a straight line in space 
containing few input data points (21 points) on line and a reference point (mid point). The set of point 
(x,y,z) lying on line goes to the first input and a second input passes the reference point (%,. Yp, Zr). The 
incorporation of the reference point provides more information to learning a system which yields better 
accuracy. Similarly, the first and second output neurons of output layer result the transformed point 
(x',y',z) on line and transformed reference point (x’,,y’;,Z,) respectively. The learning of the 
transformation is achieved by learning the algorithm presented in section 2.2 with a suitable learning rate. 
The trained QDNN is able to generalize over huge number of points cloud data of complicated geometrical 
structure like sphere, cylinder, torus and this ability of the network presents the 3D motion interpretation of 
objects. It is worthwhile to mention here that learning of phase information is not possible by RVNN hence 
such transformation is not possible through RVNN; therefore this section only presents the result obtained by 
QDNN. 


3.2.1. Similarity Transformation 

The learning of QDNN (2-6-2 model) is performed for similarity transformation, through input- 
output mapping for scaling factor 1⁄2 over the line containing 21 points, referenced in (0,0,0), as shown in 
Figure. 3(a). Convergence of mean square error (Figure. 3(b)) shows the smart learning capability of the 
proposed network. The training of QDNN with 0.00005 learning rate converges to MSE = 1.005567e-05 
after 20000 iterations. The trained network is able to generalize over many complicated standard geometric 
structures like sphere (4141 data points), cylinder (2929 data points), and torus (10201 data points) which is 
presented in Figure. 4(a), 4(b), and 4(c) respectively. 











(a) (b) 


Figure 3. (a) Training input-output mapping for scaling with scaling factor 1; 
(b) Convergence of mean square error 
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Figure 5. (a) Training patterns: input-output mapping shows transformation with scaling factor 1/2, followed 
by translation with 0.3 units in positive y-direction (b) Convergence of mean square error 


3.2.2. Scaling and translation 

The learning of 2-6-2 QDNN is performed in combination of scaling (scaling factor 1/2) and 
translation (0.3 unit in positive y-direction), through input-output mapping over the line (21 data points) and 
referenced in (0,0,0), as shown in Figure. 5(a). The convergence of QDNN in Figure. 5(b), with learning rate 
0.00005, up to 2.58514e-05 mean square error shows the smart learning capability of the proposed learning 
machine after 20000 iterations. The trained network is able to generalize well over many complicated 
standard geometric structures like sphere (4141 data points), cylinder (2929 data points), and torus (10201 
data points) as shown in Figure. 6(a), 6(b), and 6(c) respectively. 
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Figure 6. Testing results from similarity transformation through (a) sphere, (b) cylinder, and (c) torus 
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3.2.3. Scaling, translation and rotation 

The learning of QDNN for general linear transformation (scaling factor 1/2, counterclockwise 
rotation about the x-axis by 7/2 radian, and translation by (0,0,0.3)) is performed for, through input-output 
mapping over straight line and reference (0,0,0), as shown in Figure. 7(a). The 2-6-2 QDNN model is used 
for training of these transformations through 21 data points in a straight line. Convergence of mean square 
error 1.0e-04 after 20000 iterations is achieved with the 0.00005 learning rate, as shown in Figure. 7(b). The 
trained network is also able to generalize over many complicated standard geometric structures like sphere 
(4141 data points), cylinder (4141 data points), and torus (10201 data points) as shown in Figure. 8(a), 8(b), 
and 8(c) respectively. 
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Figure 7. (a) Training mapping patterns through straight line (scaling factor 1/2, counterclockwise rotated 
about the x-axis by 7/2 radian, and translated by (0,0,0.3)); (b) Square error during training of straight line 
pattern 


All transformation experiments promise the intelligent behavior of QDNN for motion interpretation of 
3D objects. Further, this novel experiment provides a direction to generalize the motion for intelligent system 
design for a variety of operations. 














(c) 


Figure 8. Generalization of a linear transformation (scaling factor 1/2, counterclockwise rotated about the x- 
axis by 7/2 radian, and translated by (0, 0, 0.3) over (a) sphere, (b) cylinder, and (c) torus 


3.3. 3D face recognition 

This section presents a basic experiment, though with a small data set but its implication is wide for 
the applicability of proposed learning machine for 3D recognition. Our method has a great deal to perform 
successful recognition in variable head position, orientation, and facial expressions. Two experiments are 
conducted here to learn and classify point cloud data of 3D faces using proposed quaternionic domain 
backpropagation algorithm. A simple structure of (1-2-1) QDNN with single input-output performs 
experiments using only two quaternionic neurons at hidden layer. 


IJAAS Vol. 7, No. 2, June 2018: 177 — 190 


JAAS ISSN: 2252-8814 o 189 


Figure. 9. Five 3D faces of same person with different orientation and poses. 


The first experiment is performed on a dataset containing 05 faces of the same person (4654 points 
cloud data) with different orientation and poses; the learning of QDNN is made with one face (Figure. 9(a)) 
and testing over all faces. Table 3 presents the testing MSE (mean square error) of all five faces which are 
comparable, hence demonstrate that they are faces of same person irrespective of variations in face 
orientation and poses. It infers straightforward learning and generalization ability of a simple QDNN which is 
not possible by RVNN. 


Table 3. Comparison of testing MSE of faces of same person with different orientation 
(MSE Training=0.0001) 


S. No. Face (Figure) Test error 
1 9(a) 2.4842¢e-04 
= 9(b) 3.543 1e-03 
3. 9(c) 5.1153e-03 
4 9(d) 4.5212e-04 
5 9(e) 3.9148e-04 


Similarly, the second experiment is performed on a dataset containing 05 faces of different people 
(6397 points cloud data); the learning of QDNN is made with one face (Figure. 10(a)) and testing over all 
faces. Table 4 presents the testing MSE of each face obtained from trained network, which shows that the 
MSE of other four faces are much higher in comparison to the face (Figure. 10(a)) used in training. This 
demonstrates that the simple QDNN correctly classifies the faces of same or different person. It again reveals 
the learning and generalization capability of a proposed learning machine where real-valued neural network 
lacks. 


(a) (b) (c) (d) (e) 


Figure. 10. Five 3D faces of different persons 


Table 4. Comparison of testing MSE of faces of different person (MSE Training = 0.0001) 


S. No. Face (Figure) Test error 
1. 10(a) 1.8214e-04 
2; 10(b) 8.1344e-01 
3. 10(c) 3.5709e-00 
4. 10(d) 6.28 14e-02 
5. 10(e) 3.1738e-01 


3. CONCLUSION 

In this paper, we present an efficient and generalized learning machine for high dimensional 
problems and evaluate it with variety of problems of different areas. The proposed neural network with 
learning algorithm in quaternionic domain directly process three or four dimension data without the hassle of 
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its different components and phase information among them. The quaternion is the number which possesses 
the magnitude of intended components and phase information of each component is embedded in it. Thus, 
quaternionic domain neural network (QDNN) leads to simple network structure, efficient learning and better 
performance; whereas conventional real-valued neural network (RVNN) deals with individual components 
hence need huge topology, slow learning and poor performance. Apart from that RVNN does not work for 
problems where it is required to learn and generalize phase information like object recognition and motion or 
transformation of objects in space. It 1s worth to mention here again that proposed machine learns the 
composition of transformations through input-output mapping over a line containing a small set of points and 
generalize this motion over complex geometrical structure such as sphere, cylinder, and torus. Although, the 
problem presented for recognition in 3D imaging is small and basic but it is very encouraging for prospective 
researcher due to network simplicity, faster convergence and the result. 
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